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Introduction 

This  document  discusses  various  problems  associated  with  preparing  good 
computer  model  documentation  and  identifies  and  describes  ways  by  which  such 
documentation  may  be  Improved.  Following  this  introductory  section  the 
following  topics  are  discussed: 

.  the  nature  of  computer  models  and  their  documentation 
.  the  relative  Importance  of  good  documentation  to  Increased 
model  use 

.  common  causes  of  poor  documentation 
.  requirements  for  producing  good  documentation 
.  constraints  to  producing  good  documentation 
Together  these  topics  address  the  problem  of  how  to  produce  better  documen¬ 
tation  to  make  more  efficient  and  effective  use  of  computer  models. 

The  views  expressed  here  have  developed  over  the  past  ten  years  from 
experience  with  the  development  and  application  of  medium  to  large  computer 
models  in  the  field  of  water  resources  engineering.  The  models  with  which 
the  author  has  been  associated  are  widely  used  nationally  and  internationally 
by  public  and  private  organizations,  and  by  practitioners  and  academics.  In 
their  field  they  are  probably  the  most  widely  used  models  in  the  world  today. 

Prepared  for  the  U.  S.  Congress,  Office  of  Technology  Assessment  Study  on 
Use  of  Models  for  Water  Resources  Management,  Planning,  and  Policy, 
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It  Is  this  author's  view  that  success  In  the  use  of  these  models  Is  due 
principally  to  the  assistance  available  to  the  user.  It  Is  assumed  the 
models  are  needed  by  professionals  In  the  water  resources  field,  and  that  they 
are  theoretically  and  technically  sound.  But  their  wide-spread  and  successful 
use  comes  from  the  fact  that  an  organization  exists  which  uses  the  models 
regularly,  and  professionals  within  that  organization  are  available  to 
assist  In  model  application.  In  addition,  these  professionals  conduct 
training  courses  on  the  models;  update,  revise  and  correct  models  and  their 
documentation;  write  and  present  technical  papers  on  model  applications. 

In  other  words  the  model,  after  development,  Is  fully  supported.  By 
contrast  most  universities  and  private  organizations  do  not  support  their 
models  after  development.  Jt  can  be  argued  that  this  is  not  their  function. 
This  is  probably  true.  Even  so,  there  must  exist  somewhere  an  organizational 
unit  to  provide  support,  otherwise  the  model  will  not  be  used. 

Good  model  documentation  Is  needed  and  necessary.  It  should  be  prepared 
for  every  model  and  It  should  be  done  well.  However,  it  Is  not  a  substitute 
for  the  assistance  of  a  professional  who  understands  the  model  and  who  Is 
using  it  regularly.  Documentation  is  not  a  substitute  for  training,  for 
technical  papers  and  reports  on  application,  for  updating,  revising,  or 
correcting  computer  code.  Continuing  support  is  central,  with  It  documentation 
has  Its  proper  place. 

Nature  of  Computer  Models  and  Their  Documentation 

When  discussing  the  success  and  failure  of  computer  models  and  the  need  for 
and  relative  Importance  of  good  documentation  it  is  of  paramount  Importance  to 


understand  the  nature  of  a  computer  model.  It  Is  an  explicit  set  of  Instruc¬ 
tions,  written  In  a  special,  machine  executable  language,  and  organized  In  a 
methodical  manner.  The  person  who  develops  a  computer  model  Is  usually  a 
technical  specialist  with  expertise  In  a  particular  discipline,  e.g., 
engineering,  and  one  who  knows  how  to  read  and  write  this  special  language. 
His  "computer  model"  Is  his  set  of  instructions  to  do  something,  for  example, 
to  simulate  the  operation  of  a  reservoir.  These  Instructions  are  his,  the 
logic  Is  his,  they  are  unique  to  this  person.  Given  the  task  to  develop  a 
computer  model  to  simulate  the  operation  of  a  reservoir  no  two  persons  will 
write  the  same  set  of  Instructions.  As  a  consequence  the  models  will  not 
compute  exactly  the  same,  nor  will  they  handle  different  conditions  the 
same.  Still,  both  may  give  answers  which  are  technically  correct.  The 
point  Is  that  the  iqyrlad  of  Instructions  which  go  Into  a  computer  model 
(around  4,000  for  a  medium  size  computer  program)  are  unique  to  the  person 
writing  those  Instructions.  And  this  poses  problems  when  It  comes  to 
documentation. 

Generally,  there  are  two  types  of  documentation:  programmer  manuals 
and  user  manuals.  Programmer  manuals  are  designed  to  assist  others  In 
understanding  the  logic  and  organization  of  the  Instructions  by  which  the 
model  operates.  This  Is  useful  when  It  Is  desired  to  change  the  Instructions 
In  some  way.  User  manuals  are  designed  to  assist  those  who  wish  to  use  the 
model  to  do  whatever  It  was  developed  to  do.  The  user  Is  not  so  much 
concerned  with  the  Instructions  and  how  they  are  organized,  but  with  how  to 
prepare  Input  to  get  a  desired  output.  Most  of  the  discussion  In  the  sections 
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.  User  documentation  requires  the  technical  specialist  who  wrote  the  model's 
instructions,  to  translate  those  machine  executable  Instructions  Into  English 
and  to  organize  this  translation  In  a  way  which,  on  the  one  hand,  communicates 
the  logic  And  organization  of  the  Instructions  and  on  the  other,  communicates 
how  to  use  the  model.  This  task  of  translation  Is  not  an  easy  one.  To  write 
Instructions  to  a  machine,  whose  response  is  known  and  predictable  requires 
one  skill.  To  translate  those  Instructions  for  a  variety  of  persons,  whose 
Interpretation  is  unpredictable  Is  quite  a  different  task.  This  Is  the  heart 
of  the  task  of  good  preparing  documentation. 


Importance  of  Good  Documentation  to  Increased  Model  Use 

Two  principal  reasons  computer  models  are  not  used  are  that  either  they 

„  i 

are  not  trusted  or  they  are  not  needed.  Since  this  conflicts  with  the  generally 

•  i 

accepted  myth  that  a  model  is  inherently  good  and  should  be  used,  a  word  of 
explanation  Is  necessary  and  will  serve  to  place  documentation  in  proper 
perspective.  In  spite  of  what  has  been  written  and  said  about  the  need  for 
and  desirability  of  computer  models  and  the  seeming  ease  with  which  they  are 
applied  they  are  nonetheless  viewed  with  a  great  deal  of  skepticism  by 
professionals  with  experience  in  the  application  of  models  to  engineering 
and  other  problems.  This  skepticism  Is  rooted  in  years  of  experience  with 
models  and  their  application.  Usually  a  new  model  doesn't  do  what  it  is 
expected  to  do.  There  can  be  many  reasons  for  this,  however,  the  professional 
working  on  a  project  is  not  about  to  entrust  the  calculations  for  some  aspect 
of  his  project  to  a  computer  model  which  Is  not  understood  or  which  produces 
questionable  results.  The  criterion  for  trustworthiness  is  an  acceptable 

r&ford  of  use  (preferably  by  those  other  than  the  model  developer)  and  the 

lfc0t  fit'  . 
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availability  of  persons  to  answer  questions  concerning  its  use.  While  good 
documentation  assists  the  user  in  understanding  the  model  and  Its  capability 
and  helps  him  to  decide  whether  it  is  needed,  it  does  not  produce  trustworthi¬ 
ness.  If  a  model  Is  needed  and  is  trustworthy,  i.e.,  will  do  what  it  Is 
suppose  to  do.  It  will  be  used  even  if  the  documentation  is  lacking.  If  a 
model  is  needed  and  has  good  quality  documentation,  but  is  not  to  be  trusted, 
it  will  not  be  used.  Thus,  good  documentation  helps  in  understanding  a  model, 
but  it  Is  not  a  principal  factor  in  its  acceptability  and  use. 

Many  models  are  not  used  because  they  are  not  needed.  Development  of  a 
model,  testing  of  a  model,  and  preparation  of  proper  documentation  does  not 
create  need  -  it  creates  a  model.  There  is  evidence  enough  that  some  models 
find  wide  acceptability  and  use  with  poor  documentation,  while  other  models 
with  seemingly  all  the  necessary  documentation  never  are  used.  One  reason  Is 
simply  that  many  models  are  not  needed. 

Two  conclusions  may  be  drawn  from  the  preceding.  First,  good  documentation 
will  never  be  a  substitute  for  model  applications  and  user  assistance  In  the 
eyes  of  the  potential  user.  Trustworthiness  is  built  upon  successful  applica¬ 
tion  and  available  support.  Second,  good  documentation  does  not  create  need. 
The  user,  functioning  in  the  free  market,  will  welcome  models  which  are 
trustworthy  and  help  in  solving  problems.  Lack  of  acceptance  does  not  mean 
this  welcome  has  been  withdrawn. 

Common  Causes  of  Poor  Documentation 

Five  common  causes  of  poor  computer  model  documentation  are  identified 
below. 
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.  When  models  are  developed  by  government  contract  frequently 
the  organization  funding  the  development  does  not  have  the 
organizational  unit  to  take  responsibility  for  its  operation 
and  maintenance  or  is  unwilling  to  allocate  time  and  staff 
for  such  support.  Such  negligence  and  lack  of  capability  for 
quality  control  encourages  and  tolerates  poor  documentation. 

.  It  is  a  difficult,  time  consuming  task  to  translate  instructions 
and  logic  from  a  machine  executable  language  to  English  in  a  way 
which  clearly  communicates  how  a  model  operates  and  how  It  should 
be  used. 

.  The  person  who  writes  the  machine  instructions  may  not  have  the 
patience,  ability,  or  interest  to  translate  and  interpret  these 
instructions  into  English. 

.  Good  documentation  is  not  as  common  as  poor  documentation. 
Consequently  the  person  who  writes  documentation  may  not  be  aware 
of  what  constitutes  good  documentation. 

.  Inadequate  time,  funds  and  staff  are  commonly  allocated  to  the 
documentation  task. 

Completing  development  of  a  computer  model  may  be  viewed  as  the  beginning 
or  the  end.  An  organization  which  has  a  unit  to  support  and  maintain  a  newly 
developed  model  will  view  its  development  as  the  beginning  -  the  beginning  of 
its  use,  its  application,  its  growth  In  capability.  As  a  consequence  there 
will  be  greater  incentive  for  preparing  good  documentation  to  support  the  long 
term  commitment  to  the  model.  An  organization  which  does  not  have  a  unit  for 
support  and  maintenance,  or  which  may  have  such  a  unit  but  does  not  assume 
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the  responsibility,  will  view  its  development  as  the  end.  While  it  may  be 
hoped  that  the  model  is  picked  up  and  used  by  others,  no  commitment  of 
resources  (time,  staff,  funds)  is  made.  In  this  situation,  there  is  less 
Incentive  to  prepare  good  documentation  since  development  of  the  model  and  ' 
associated  documentation  complete  the  work. 

The  difficulty  of  preparing  good  documentation  should  not  be  under¬ 
estimated.  A  medium  size  computer  model  will  consist  of  several  thousand 
explicit  instructions.  To  properly  use  the  model  the  user  must  be  provided 
with  documentation  which  provides:  (1)  clear  instruction  on  how  to  prepare 
input  data  such  that  each  machine  instruction  is  executed  properly,  (2)  a 
clear  understanding  of  the  physical,  engineering,  mathematical,  biological 
etc.  processes  or  methods  which  are  used  in  the  model,  (3)  a  clear  under¬ 
standing  of  the  model  output  and  how  that  output  relates  to  the  phenomena 
being  modeled,  and  (4)  a  clear  knowledge  of  how  the  model  will  respond  to 
different  combinations  of  input  instructions  and  study  conditions.  In  the 
world  of  computer  models,  close  is  not  good  enough.  The  computer  demands 
(and  gets)  exactness.  The  user's  input  data  cannot  be  almost  correct. 

The  machine  executable  instructions  cannot  be  nearly  complete.  The  task 
of  preparing  good  documentation  is  one  of  bringing  to  the  user  a  clarity 
of  thought,  understanding,  and  knowledge  such  that  precise  instructions  can 
be  given  and  the  model's  response  will  be  as  desired  and  expected. 

With  regard  to  the  third  problem,  the  person  (or  persons)  who  have 
written  the  machine  Instructions  for  a  model  really  have  no  need  for 
documentation  other  than  as  a  reminder  of  what  they  may  forget.  Documen¬ 
tation  is  principally  for  others.  The  person  writing  the  machine  instructions 
knows  what  they  are,  what  they  are  intended  to  do  and  how  they  are  organized. 


Questions  or  problems  can  be  readily  answered.  It  is  difficult  then  to  develop 
the  patience  to  place  oneself  in  the  user's  role  and  communicate  all  that  is 
needed  to  be  known  and  understood  about  the  model.  In  addition,  such 
comnuni cation  requires  writing  abilities  different  from  those  required  for 
the  computer.  To  prepare  instructions  in  a  machine  executable  language  for 
a  digital  computer  is  quite  a  different  task  than  writing  in  English  for 
people.  While  each  requires  logic  and  organization,  their  nature  is  quite 
different.  Some  people  can  do  both,  many  can  do  only  one  or  the  other. 

Also,  it  is  a  special  skill  to  be  able  to  write  in  English  without  using 
excessive  computer  jargon  which  may  obscure  the  real  meaning.  Lastly,  is 
the  question  of  interest.  Clearly,  the  rewards  in  both  public  and  private 
practice  are  for  computer  model  development,  not  for  post  facto  documentation. 
Such  rewards  are  generally  professional  (papers  published)  and  economic 
(projects  completed). 

There  is  a  need  for  more  examples  of  good  documentation.  The  basic 
requirements  for  good  documentation  will  be  discussed  in  the  next  section. 

Like  a  quality  crafted  chair,  it  is  more  than  four  legs,  a  seat,  and  back, 
which  makes  it  a  quality  product  -  it  is  the  workmanship  and  materials 
which  go  into  it.  Likewise  for  model  documentation. 

It  is  fair  to  say  that  most  computer  model  development  takes  longer, 
takes  more  funds,  and  is  more  complex  than  estimated  at  the  beginning. 

The  additional  time,  funds,  and  staff  are  often  taken  from  that  allocated 
to  documentation.  This  creates  an  atmosphere  of  pressure  and  shortage  of 
of  time  where  patience  is  needed.  Under  such  conditions  documentation  can 
be  prepared,  even  documentation  meeting  specified  standards.  However,  It 
is  usually  not  good,  well  thought-out,  clearly  communicated  documentation. 


Producing  Good  Documentation 


Organizational  Support.  There  is  no  substitute  for  having  an  organiza¬ 
tional  unit  which  is  responsible  for  a  model's  development  and  documentation, 
and  is  responsible  for  its  continuing  use  and  maintenance.  The  technical 
quality  of  documentation  (in  contrast  to  visual  quality)  can  only  be 
assessed  through  model  use.  For  models  developed  by  contract  such  a  unit 
can  work  with  the  contractor  during  the  contract  to  guide  and  evaluate  the 
documentation.  Final  payment  can  and  should  be  withheld  pending  '  >pletion 
of  acceptable  documentation.  This  will  require  testing  and  usinc  >e  model. 

Knowing  that  a  model  and  its  documentation  will  be  thoroughly  ;d  and 
evaluated  by  competent  technical  specialists  under  the  contract  w  ‘  ->e  a 
strong  incentive  for  the  person  who  wrote  the  machine  instructions  to  care¬ 
fully  communicate  the  necessary  information  to  the  user.  It  will  be  an 
incentive  to  develop  the  necessary  patience  and  interest  in  the  documentation 
task.  During  the  review  process  concepts,  instructions  and  examples  which 
are  not  clear  can  be  clarified  in  the  documentation.  As  discussed  under  the 
causes  of  poor  documentation  the  principal  task  of  the  person  who  wrote  the 
machine  instructions  is  to  translate  these  instructions  into  English  for  the 
user.  When  a  competent  user  is  available  under  the  contract  to  test  and 
evaluate  these  instructions,  their  adequacy  can  be  evaluated  and  improvements 
made. 

For  models  not  developed  by  contract  but  developed  within  an  organization 
the  problem  is  more  difficult.  Here  peer  and  organization  review  are  necessary. 
However,  as  model  development  nears  completion  competition  for  time  and  staff 
become  acute.  In  this  case  it  is  important  that  the  model  be  applied  by 
others.  As  the  model  is  used  the  developer  will  receive  peer  fee  ack  on 


the  adequacy  of  the  work,  and  this  hopefully  will  lead  to  improvement.  This 
is  probably  the  best  way  to  encourage  good  documentation. 

Documentation  Content.  When  there  is  a  need  for  information  on  model 
use  and  it  is  not  covered,  or  is  inadequately  covered,  in  the  documentation 
the  potential  user  has  three  principal  options:  (1)  contact  the  person  who 
wrote  the  machine  instructions  for  the  model,  (2)  attempt  to  decipher  the 
machine  instructions,  or  (3)  not  use  the  model.  The  first  may  not  be 
possible,  the  second  is  time  consuming,  and  the  third  is  to  be  avoided. 
Consequently,  it  is  important  that  the  documentation  be  complete,  clear, 
and  accurate.  The  following  is  a  list  of  essential  information  which  should 
be  included  in  user  documentation: 

.Introduction 

.  Theory  and  Computational  Methods 
.  Model  Capabilities 
.  Data  Requirements 
.  Input  Specifications 
.  Output  Description 
.  Example  Applications 

The  “Introduction"  should  present  information  on  the  origin  and  author 
of  the  model,  when  it  was  developed,  an  overview  of  its  capabilities  and 
limitations,  computer  equipment  requirements,  the  person  and  organization 
responsible  for  support,  and  other  general  information  of  importance  to  the 
user. 

"Theory  and  Computational  Methods"  should  describe  the  engineering, 
economic,  biological,  etc.  theory  or  theories  used  in  the  model  or  if  they 
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are  commonplace  in  the  profession  appropriate  references  may  be  cited.  This 
should  include  a  complete  description  of  the  equations,  notation,  and  principles 
used.  Frequently  various  mathematical  or  statistical  methoas  are  used  in  the 
computations.  These  should  be  identified  and  described  or  references  cited. 

The  general  computational  procedure,  i.e.,  the  order  of  computation,  should 
also  be  described. 

"Model  Capabilities"  should  describe  what  the  model  is  designed  to  do  and 
what  it  will  not  do,  if  this  is  not  obvious.  This  will  include  both  basic 
capabilities  and,  as  is  common  in  more  complex  models,  optional  capabilities. 

A  section  describing  "Data  Requirements,"  written  in  the  context  of  the 
engineering,  economic,  biological  phenomenon  being  modeled  can  be  most  useful 
to  the  user.  With  both  the  theory  and  model  capabilities  set  forth,  the  user 
is  directed  to  the  data  required  by  the  theory  to  produce  the  desired  results. 
This  data  description  is  different  from  the  input  specifications. 

"Input  Specifications"  describe  how  the  user  should  prepare  data  to 
properly  meet  the  machine  executable  instructions.  The  data  requirements 
mentioned  above  were  in  the  context  of  the  theory  and  capability,  i.e.,  in 
the  context  of  the  professional  discipline.  When  preparing  input  specifications 
these  data  are  put  into  a  form  which  is  acceptable  to  the  machine  instructions. 
Here  precision  is  critical  for  proper  execution  of  the  program. 

"Output  Description"  should  provide  a  description  and  explanation  of  all 
output  from  the  model.  This  should  include  an  explanation  of  all  terms, 
abbreviations  and  notations.  Units  and  time  periods  should  be  described  and 
all  output  devices-printer,  tape,  CRT,  etc.  should  be  discussed. 


"Example  Applications"  are  most  important.  In  the  examples  the  theory, 
capability,  data  requirements,  input  anc  utput  are  all  illustrated.  Examples 
over  a  wide  range  of  applications  should  be  selected.  Each  example  should  be 
clearly  organized  with  textual  discussion  from  theory  to  output.  Examples 
should  also  be  selected  to  allow  validation  of  the  model  when  it  is  used  on 
a  computer  different  from  the  one  on  which  it  was  developed. 

The  foregoing  is  a  brief  description  of  the  basic  content  of  user  docu¬ 
mentation.  These  items  are  an  essential  and  necessary  part  of  good  documentation. 
Even  so,  their  inclusion  does  not  insure  quality.  One  could  discuss  each  of 
these  topics  and  still  produce  poor  documentation.  A  knowledge  of  what  should 
be  included  in  good  documentation  must  be  coupled  with  motivation,  ability  and 
time  to  prepare  it. 

Constraints  to  Producing  Good  Documentation 

As  discussed  previously  the  principal  need  to  produce  good  documentation 
is  continuing  organizational  support  and  maintenance  of  the  developed  model. 

The  principal  constraint  is  the  lack  of  such  organizational  support  and  the 
associated  fixing  of  on-going  responsibility  for  the  model.  Models  are 
frequently  developed  by  universities  and  private  contractors  for  government 
organizations,  however,  there  is  no  organizational  unit  to  use  and  maintain 
the  model,  thus.  It  doesn't  take  long  for  the  model  (and  taxpayers  investment) 
to  get  "shelved."  Yet  this  same  model  may  appear  in  the  literature  and  give 
the  "appearance"  of  being  operational.  Yet  the  only  source  of  assistance 
is  the  documentation  developed  with  the  model. 
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Overcoming  this  constraint  is  both  easy  and  difficult.  It  is  easy  in 
that  all  that  is  needed  prior  to  development  of  the  model  is  that  the 
responsibility  for  its  on-going  application  and  maintenance  be  assigned  to 
a  person  and  organizational  unit.  And,  that  this  information  be  made 
available  with  the  model.  It  is  difficult  to  overcome  in  that  there  can 
be  a  "paper"  designation  of  responsibility  and  a  token  allocation  of 
resources  for  use  and  support.  Probably  the  best  approach  is  not  to  allow 
model  development  unless  it  is  fully  supported  and  maintained  by  the  sponsor 
of  the  model. 

There  are  no  major  constraints  to  documentation  content  whether  by 
standards  or  some  other  means.  The  major  elements  of  good  documentation 
are  well  known,  some  examples  exist,  and  they  can  be  required  in  any 
government  contract  or  within  the  government  by  any  agency.  Those  for 
whom  models  are  to  be  developed  simply  must  desire  that  it  be  done  properly. 

Summary  and  Conclusions 

The  following  points  have  been  discussed  to  provide  an  understanding 
of  some  of  the  important  causes  of  poor  documentation  and  to  suggest  ways 
by  which  documentation  can  be  improved. 

.  A  computer  model  is  an  explicit  set  of  instructions  written  in 
a  special,  machine  executable  language.  Documentation  is  a 
translation  of  these  instructions  into  English  such  that  a 
user  is  provided  with  a  clear  understanding  of  the  model  and 
its  use.  Such  a  translation  requires  skill,  patience  and 
interest. 
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.  The  principal  reasons  computer  models  are  not  used  are  that 
either  they  are  not  trusted  or  they  are  not  needed.  Good 
documentation  is  of  secondary  importance  until  trust  and  need 
are  established. 

.  Several  causes  of  poor  documentation  include:  the  absence  of 
an  organizational  unit  to  provide  on-going  support  and  maintenance; 
the  difficulty  of  the  task  of  preparing  good  documentation;  the 
lack  of  writing  ability  on  the  part  of  the  model  developer; 
a  definition  of  what  constitutes  good  documentation;  Inadequate 
time,  funds  and  staff. 

.  The  principal  needs  for  producing  good  documentation  are: 
an  organizational  unit  which  Is  responsible  for  model  development, 
documentation,  and  continuing  use  and  maintenance;  Identification 
of  the  essential  information  which  should  be  Included  in  user 
documentation;  a  model  developer  who  has  the  desire,  ability 
and  time  to  prepare  good  documentation. 

.  The  principal  constraint  to  producing  good  documentation  is  the 
establishment  of  responsible  organizational  units  within  each 
agency  to  support  and  use  developed  models. 

The  development  of  a  computer  model  should  be  viewed  as  the  beginning  of 
a  new  tool,  a  new  technology,  a  new  capability.  Documentation  Is  Intended 
to  assist  future  users  In  the  application  of  the  model.  However,  documentation 
can  never  be  successful  by  itself.  On-going  technical  and  organizational 
support  are  needed.  Someone  must  be  responsible  for  its  future. 
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